Win64 LLP64 ABI support with cross-compiler tooling by xdqi · Pull Request #630 · lkl/linux

xdqi · 2026-05-04T13:42:34Z

lkl: Win64 LLP64 ABI support with cross-compiler tooling

Summary

This PR enables LKL to be compiled for Win64 (PE) targets using a hybrid LP64/LLP64 toolchain. The core challenge is that Win64 follows the LLP64 data model (long=32-bit) while the LKL kernel internally uses LP64 (long=64-bit). These patches bridge that gap with compile-time typedef remapping, cross-compiler shims, and a custom setjmp/longjmp for the MS ABI.

Changes

1. `__lkl_long_t` typedef for cross-ABI compatibility

New header arch/lkl/include/uapi/asm/lkl_long.h defines __lkl_long_t / __lkl_ulong_t:
- LLP64 (MinGW-w64 x64): expands to long long (64-bit)
- LP64 (kernel, Linux, Cygwin, macOS): expands to long (unchanged)
headers_install.py auto-replaces long with __lkl_long_t in installed UAPI headers and inserts the required include
All host_ops implementations, syscall parameters, ioctl casts, and tests updated accordingly

2. Cross-compiler wrapper scripts (`tools/lkl/bin/`)

x86_64-w64-mingw32-gcc: routes kernel code (-D__KERNEL__) to x86_64-pc-cygwin-gcc (LP64), user-space to real MinGW-w64 gcc (LLP64), with generated specs to a patched linker
x86_64-w64-mingw32-cc: symlink to the gcc wrapper
Patched x86_64-w64-mingw32-ld: supports PE --image-base=0x10000
Patched x86_64-w64-mingw32-objcopy: PE format support

3. Custom setjmp/longjmp for x86_64 MS ABI

Replaces libc setjmp.h with raw register save/restore inline assembly
Bypasses Windows SEH RtlUnwindEx which cannot handle cross-stack jumps used by LKL's cooperative context switching
Falls back to libc setjmp/longjmp on non-Win64 platforms

4. Patched binutils: fix weak externals in PE COFF after `ld -r`

When ld -r creates a relocatable object (e.g. vmlinux), the tagndx field in PE COFF weak external aux entries is not relocated — it retains stale symbol indices from the original object. All 352 C_NT_WEAK symbols in the LKL vmlinux have incorrect tagndx values: some point to non-code sections (e.g. .data), others to completely unrelated code symbols (e.g. bpf_arena_get_user_vm_start → cmd_line_append).

The existing binutils code only fell back to name-based lookup when tagndx pointed to a non-code section, missing the case where it points to a valid-but-wrong code symbol. This caused objcopy -G to resolve weak externals to incorrect addresses, leading to runtime crashes (e.g. ktime_get dereferencing NULL through tk_core->clock).

The fix in coff_nt_weak_to_local() (lkl-binutils@64f5155) always searches for the .weak.<name>.<tag> implementation by name first — PE COFF weak external naming is deterministic and reliable. Only falls back to tagndx when no name match is found.

Repositoriy: xdqi/lkl-binutils (lkl branch, based on binutils-2.25.1)

Toolchain

The build environment is Debian 13 (trixie):

Component	Source
MinGW-w64 cross-compiler	Debian 13 system packages (`gcc-mingw-w64-x86-64`, `binutils-mingw-w64-x86-64`)
Cygwin cross-compiler	deb-cygwin snapshot — `x86_64-pc-cygwin-gcc` produces LP64 PE/COFF output directly
Patched binutils	Custom ld/objcopy with PE `--image-base` support (lkl-binutils)

The kernel is compiled with Cygwin GCC (LP64, direct PE output), while user-space libs and tests use MinGW-w64 GCC (LLP64). The __lkl_long_t typedef bridges the ABI between these two worlds.

Disclaimer

This pull request is mostly done by Claude Code, but I audited every line of code :).

Cross-compiler wrapper (tools/lkl/bin/): - x86_64-w64-mingw32-gcc: routes kernel code (-D__KERNEL__) to x86_64-pc-cygwin-gcc (LP64), user-space to real mingw-gcc (LLP64) with a specs file to override linker path - x86_64-w64-mingw32-cc: symlink to gcc wrapper - x86_64-w64-mingw32-ld: patched ld supporting PE --image-base - x86_64-w64-mingw32-objcopy: patched objcopy for PE format Custom runtime: - lib/jmp_buf.c: setjmp/longjmp for x86_64 MS ABI (10 callee-saved XMM registers + different GPR set vs SysV)

PE64 default image base is 0x140000000. Weak symbols resolve to 0, causing rel32 relocation overflow (displacement > 2GB). Setting --image-base=0x10000 keeps all addresses in the low 32-bit range.

On Win64 (MinGW-w64 LLP64), 'long' is 32-bit while the LKL kernel (compiled with Cygwin LP64 GCC) uses 64-bit 'long'. This causes ABI mismatches in syscall parameters, UAPI struct layouts, and pointer casts between user-space and kernel. Introduce __lkl_long_t / __lkl_ulong_t conditional typedefs (arch/lkl/include/uapi/asm/lkl_long.h): - LLP64 (MinGW-w64 x64): expands to 'long long' (64-bit) - LP64 (Linux/Cygwin/macOS): expands to 'long' (no change) headers_install.py replaces 'long' with __lkl_long_t in installed UAPI headers and auto-inserts #include <lkl/asm/lkl_long.h>. Source files updated to use __lkl_long_t for syscall parameter arrays, ioctl pointer casts, and return values. Also fix case-sensitive includes in virtio_net_wintap.c for Linux cross-compilation.

tavip

Thanks for the PR @xdqi ! Could you please expand a little bit about the benefits of switching to LLP64?

tavip · 2026-05-04T18:19:10Z


 define nt_host
  $(call set_autoconf_var,NT,y)
+  $(call set_kernel_config,INIT_STACK_ALL_ZERO,n)


Why is this needed?

It's used to make my old Cygwin GCC 11 happy. We can remove it if we maintain our own newer Cygwin cross compiler.

xdqi · 2026-05-05T11:57:19Z

Just to make the LKL run under vanilla 64-bit Win32 environment instaed of Cygwin.
I used the Cygwin compiler to compile kernel because kernel have to work in LP64.
But the Win32 is LLP64, so the bridging code (LKL API for user) must use a special LKL long type.

Add missing SPDX-License-Identifier to the cross-compiler wrapper script and fix missing blank line after declaration in jmp_buf.c. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

Remove leading double underscores from the cross-ABI typedefs. These types bridge the LP64/LLP64 gap between kernel and host userspace and do not need reserved-namespace identifiers. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

lkl.h uses lkl_long_t / lkl_ulong_t but does not include the header that defines them. Add the missing include so native Linux builds can find these typedefs. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

Replace the i686 cross-binutils (as, ld, objcopy) with freshly built versions from binutils-gdb configured with --target=i686-w64-mingw32. These properly handle 32-bit PE format and fix the page fault crash during kernel boot on mingw32. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

CONFIG_LKL_HOST_MEMCPY/MEMSET/MEMMOVE are enabled in the kernel config, so init.c expects these slots in lkl_host_ops to be wired up. Without them lkl_init() returns -1 and the kernel never starts on Windows. Wrap the libc routines so the lkl_ulong_t signature matches.

Two changes: 1) Any invocation that ends up linking objects produced by the kernel build (which were compiled with cygwin-gcc for LP64 + working REL32 range) must also link with cygwin-gcc — MinGW ld can't resolve the cygwin objects. Detect this by the absence of -c/-S/-E and by *lkl.o on the command line. 2) Use \`which\` lookups instead of hardcoded /usr/bin paths so the wrapper works on hosts where the toolchains live elsewhere.

thehajime · 2026-05-18T00:35:07Z

@xdqi thanks for the patchset and sorry for the late response (resulting required to rebase your branch..).
I'll look into details to your patches very soon.

one quick question: do you think it's better to have a separate test matrix other than current msys2, or drop the msys2 to replace with this patchset ? I'd be fine either case.

thehajime · 2026-05-18T05:46:13Z

@xdqi read through the patches briefly.
most of them, including lkl_long_t typedefs, looks fine for me.

I'd like to test this new platform over gh actions (or any CI) during any commits, because it's very hard to track down which commits break this feature, especially the platform is non-Linux one. It would be nice to include a test update in this patchset if possible.

xdqi · 2026-05-18T05:55:03Z

@thehajime
I prefer to go with Win64 test matrix instead of MSYS ones.
For the tests, I tried to build a cross toolchain from Linux to all MSYS2 targets, see https://github.com/xdqi/msys-cross/blob/main/scripts/build.sh . But I haven't deeply gone through current test pipelines in lkl/linux repo. I'll try to add two tests with MINGW64/MSYS cross toolchains + wine under Linux, and MSYS gcc and MINGW64 gcc under MSYS2.

xdqi added 3 commits May 4, 2026 17:52

lkl: add --image-base=0x10000 for Win64 PE builds

baaf3c6

PE64 default image base is 0x140000000. Weak symbols resolve to 0, causing rel32 relocation overflow (displacement > 2GB). Setting --image-base=0x10000 keeps all addresses in the low 32-bit range.

tavip reviewed May 4, 2026

View reviewed changes

tavip requested a review from thehajime May 4, 2026 18:44

xdqi added 4 commits May 6, 2026 13:43

lkl: add SPDX tag to gcc wrapper and fix coding style

b951be6

Add missing SPDX-License-Identifier to the cross-compiler wrapper script and fix missing blank line after declaration in jmp_buf.c. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

lkl: include lkl_long.h in lkl.h for lkl_long_t definition

2a90ef7

lkl.h uses lkl_long_t / lkl_ulong_t but does not include the header that defines them. Add the missing include so native Linux builds can find these typedefs. Signed-off-by: Sheldon Qi <3365420+xdqi@users.noreply.github.com>

xdqi force-pushed the lkl-win64 branch from 8f750f3 to fec8f7a Compare May 7, 2026 06:36

xdqi added 2 commits May 14, 2026 02:10

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Win64 LLP64 ABI support with cross-compiler tooling#630

Win64 LLP64 ABI support with cross-compiler tooling#630
xdqi wants to merge 9 commits into
lkl:masterfrom
xdqi:lkl-win64

xdqi commented May 4, 2026

Uh oh!

tavip left a comment

Uh oh!

Uh oh!

tavip May 4, 2026

Uh oh!

xdqi May 5, 2026

Uh oh!

Uh oh!

xdqi commented May 5, 2026

Uh oh!

thehajime commented May 18, 2026

Uh oh!

thehajime commented May 18, 2026

Uh oh!

xdqi commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

xdqi commented May 4, 2026

lkl: Win64 LLP64 ABI support with cross-compiler tooling

Summary

Changes

1. __lkl_long_t typedef for cross-ABI compatibility

2. Cross-compiler wrapper scripts (tools/lkl/bin/)

3. Custom setjmp/longjmp for x86_64 MS ABI

4. Patched binutils: fix weak externals in PE COFF after ld -r

Toolchain

Disclaimer

Uh oh!

tavip left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tavip May 4, 2026

Choose a reason for hiding this comment

Uh oh!

xdqi May 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

xdqi commented May 5, 2026

Uh oh!

thehajime commented May 18, 2026

Uh oh!

thehajime commented May 18, 2026

Uh oh!

xdqi commented May 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

1. `__lkl_long_t` typedef for cross-ABI compatibility

2. Cross-compiler wrapper scripts (`tools/lkl/bin/`)

4. Patched binutils: fix weak externals in PE COFF after `ld -r`